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In  this  paper,  we  propose  a  unified  cooperative  control  architecture  (UCCA)  that  supports  effective  cooperation  of 
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other  function  modules  through  well  defined  interfaces;  ii)  it  allows  the  efficient  sharing  of  UAV  control  and  onboard 
resources  by  the  function  modules  and  is  able  to  effectively  handle  simultaneously  multiple  objectives  in  the  UAV 
operation;  iii)  it  facilitates  the  cooperation  among  different  function  modules;  iv)  it  supports  effective  cooperation 
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ABSTRACT 

In  this  paper,  we  propose  a  unified  cooperative  control  architecture  (UCCA)  that  supports  effective  cooperation  of 
Unmanned  Aerial  Vehicles  (UAVs)  and  learning  capabilities  for  UAV  missions.  Main  features  of  the  proposed  UCCA 
include:  i)  it  has  a  modular  structure;  each  function  module  focuses  on  a  particular  type  of  task  and  provide  services  to 
other  function  modules  through  well  defined  interfaces;  ii)  it  allows  the  efficient  sharing  of  UAV  control  and  onboard 
resources  by  the  function  modules  and  is  able  to  effectively  handle  simultaneously  multiple  objectives  in  the  UAV 
operation;  iii)  it  facilitates  the  cooperation  among  different  function  modules;  iv)  it  supports  effective  cooperation  among 
multiple  UAVs  on  a  mission’s  tasks,  v)  an  objective  driven  learning  approach  is  also  supported,  which  allows  UAVs  to 
systematically  explore  uncertain  mission  environments  to  increase  the  level  of  situation  awareness  for  the  achievement 
of  their  mission/task  objectives. 

Keywords:  UAV  cooperative  control  architecture,  information  fusion,  sensor  management 


1.  INTRODUCTION 

For  future  surveillance,  remote,  and  hazardous  missions,  organizations  will  rely  on  unmanned  aerial  vehicles  (UAVs)  to 
collect  social  and  environmental  data  of  ground  conditions.  To  enable  a  multitude  of  UAVs  to  operate  in  a  single 
airspace,  research  in  the  20th  century  focused  on  reactive  collision  avoidance,  intelligent  path  planning,  and 
sensor/resource  scheduling.  Learning  methods  (e.g.  reinforcement)  and  Markov  Decision  Process  (MDPs)  were  trained 
offline  to  optimize  a  UAV  routes  and  tasks;  however  these  solutions  were  subject  to  en-route  challenges  of  re-planning 
do  to  mission  changes  and  unforeseen,  real-time  mission  alterations.  Assuming  individual  UAVs  could  not  achieve 
complete  autonomy;  efforts  in  the  21st  century  focused  on  swarms  of  UAVs  working  cooperatively.  Cooperative  UAV 
strategies  partitioned  the  individual  UAV  tasks  to  the  group  using  similar  methods  as  for  single  UAVs,  with  significant 
efficiency  benefits  of  “cooperative”  identification  of  communication  sharing.  Key  to  future  implementation  and 
operational  use  of  intelligent  and  cooperative  control  methods  requires  (1)  robust  control  and  optimization  to  changing 
physical  sensor,  target,  and  environmental  conditions,  (2)  understanding  of  mission  goals  do  to  adversarial  deceptions 
using  functional  game-theoretical  control ,  and  (3)  situational  awareness  control  to  manage  uncertainty,  agent-based  task 
requirements,  and  global/local  performance  objectives.  To  meet  these  challenges,  in  this  paper,  we  propose  an  unified 
cooperative  control  architecture  (UCCA)  as  a  general  framework  to  support  effective  cooperation  of  a  group  or  groups  of 
UAVs  and  learning  capabilities  in  dynamic,  uncertain  mission  environments.  Key  advantages  of  the  proposed  UCCA 
include 

1)  The  proposed  UCCA  for  UAV  missions  provides  a  unified  framework  for  various  function  modules  to  effectively 
operate  and  cooperate.  The  UCCA  has  a  concise  three  layer  structure,  which  includes  a  physical  layer,  a  function 
layer,  and  a  situation  awareness  layer.  The  physical  layer  defines  the  basic  control  space  of  the  UAV  and  onboard 
resources,  and  provides  control  interfaces  to  the  function  layer.  The  function  layer  consists  of  function  modules  that 
handle  various  tasks  in  UAV  missions  and  a  special  Task  Coordination  Module  (TCM)  that  allows  other  function 
modules  to  cooperate  and  share  the  control  of  the  UAV  and  its  onboard  resources  [1].  The  situation  awareness  layer 
is  in  place  to  allow  the  effective  sharing  of  information  among  different  function  modules.  In  the  proposed  UCCA, 
new  function  modules  can  be  easily  incorporated  and  cooperate  with  existing  function  modules. 
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2)  A  novel  Hierarchical  Layered  Decision  Framework  (HLDF)  is  adopted  in  the  TCM  of  the  function  layer  for  the 
sharing  of  UAV  control  and  onboard  resources  among  function  modules  for  multiple  competing  objectives.  The 
HLDF  suits  perfectly  the  modular  structure  of  the  function  layer.  Unlike  conventional  combined  objective 
approaches  for  handling  multiple-objectives,  the  HLDF  allows  the  specification  of  desired  performance  for 
individual  objectives  for  different  function  modules,  and  is  able  to  yield  Pareto  optimal  control  decisions  with  the 
desired  overall  performance.  An  objective  with  a  higher  importance  will  be  protected  from  possible  compromises 
from  other  less  important  ones.  And  depending  on  the  nature  of  the  objective,  suitable  control  strategies,  e.g.,  one- 
step  or  multi-step  look-ahead  control  strategies,  etc.,  can  be  used  in  different  function  modules  [1]. 

3)  The  proposed  UCCA  provides  excellent  support  for  the  cooperative  control  of  a  team  of  UAVs  for  complex 
missions.  For  UAV  cooperation,  a  networking  module  will  be  implemented  in  the  function,  which  is  dedicated  to 
maintaining  a  communication  network  among  the  group  of  UAVs  and  provide  communication  services  to  other 
function  modules.  The  proposed  UAV  UCCA  also  facilitates  the  integration  of  higher-level  function  modules,  such 
as  coordinate  mission  planning,  and  allows  them  to  cooperate  effectively  with  existing  lower-level  function 
modules.  The  above  supports  enable  function  modules  at  multiple  UAVs  to  efficiently  cooperate  on  all  levels  of 
UAV  mission/tasks. 

4)  An  objective  driven  learning  (ODL)  framework  is  supported  to  allow  UAVs  to  systematically  increase  the  level  of 
situation  awareness  for  the  achievement  of  their  mission/task  objectives.  In  the  proposed  UCCA,  uncertainties  of  the 
environment  and  adversaries  are  taken  into  account  by  the  uncertainties  of  the  states  and  environmental  parameters 
in  the  situation  awareness  layer.  Algorithms  will  be  developed  for  various  function  modules  to  allow  the  evaluation 
of  the  impact  of  uncertainties  to  the  achievement  of  their  objectives.  Based  on  the  evaluation,  function  modules  are 
able  to  collect  necessary  information  through  on  board  sensors,  other  function  modules,  and  the  networking  module 
from  other  UAVs  to  maximize  the  system  performance. 

The  paper  is  organized  as  follows.  Section  2  introduces  the  proposed  UCCA,  its  three-layer  structures,  and  basic 
components.  Section  3  and  4  discuss  the  UCCA’s  supports  for  cooperative  control  of  a  group  of  UAVs  and  the  objective 
driven  learning  framework.  Section  5  summarizes  the  paper  with  concluding  remarks. 


2.  THE  PROPOSED  UCCA  FOR  UAV  MISSIONS 

The  proposed  UCCA  for  UAV  missions  provides  a  unified  framework  for  various  function  modules  to  effectively 
operate  and  cooperate.  It  is  designed  to  provide  efficient  support  for  UAV  multi-tasking,  cooperation,  and 
responsiveness  of  a  network  of  UAVs  to  mission  uncertainties.  Fig.  1  shows  the  structure  of  the  UCCA.  It  consists  of 
three  interconnected  layers  which  are  a  physical  layer,  a  function  layer,  and  a  situation  awareness  layers. 


Figure  1.  Structure  of  the  UAV  system  in  the  unified  cooperative  control 

architecture 


The  physical  layer  supports  the  basic  operations  of  the  UAV  and  the  onboard  hardware  resources.  It  consists  of  the 
basic  control  systems  for  the  UAV  and  other  on  board  hardware  resources,  such  as  sensors,  communication  system, 
weapons,  etc.  In  the  proposed  UCCA,  the  physical  layer  defines  the  control  space  of  the  UAV  and  onboard  resources, 
and  provides  interfaces  that  allow  the  access  from  the  function  layer. 

The  function  layer  (indicated  by  the  dashed  line  areas)  consists  of  function  modules  that  handle  various  tasks  in  UAV 
missions,  e.g.,  safe  navigation,  collision  avoidance,  path  planning,  search,  target  identification,  target  tracking,  etc.  Each 
function  module  is  specialized  in  a  specific  type  of  UAV  task  and  has  its  own  model  of  the  problem  using  a  distinct  set 
of  states  and  parameters,  and  a  Task  Coordination  Module  (TCM)  that  allows  other  function  modules  to  coordinate  and 
cooperate. 

In  the  proposed  UCCA,  states/parameters  of  a  function  module  can  be  shared  with  other  function  modules  by  registering 
them  at  the  situation  awareness  layer  (SAL).  The  SAL  summarizes  the  current  knowledge  of  the  UAV  about  the 
environment  from  different  task  perspectives.  Structures  of  the  SAL  and  the  function  layer  will  be  further  discussed  next. 

2.1  The  situation  awareness  layer 

In  the  proposed  UCCA,  the  SAL  is  in  place  to  facilitate  information  sharing  among  different  function  modules.  It 
consists  of  states  registered  by  the  function  modules  which  summarize  the  current  knowledge  of  the  UAV  about  the 
environment  from  different  tasks’  perspectives.  Fig.  2  shows  an  illustrative  example  of  states/parameters  registered  in 
the  SAL  and  their  relationships  with  function  modules  in  the  function  layer.  As  shown  in  Fig.  2,  registered  states  from  a 
function  module  can  be  accessed  by  other  function  modules  as  their  inputs.  Together  states  registered  states  at  the  SAL 
form  a  directed  graph,  which  can  be  analyzed  using  statistic  models  such  as  Bayesian  networks  and  inference  diagrams 
[2]  to  evaluate  the  impact  of  the  performance  of  one  function  module  on  that  of  other  function  modules  and  on  the 
overall  system  performance.  The  structure  also  facilitates  the  introduction  of  new  function  modules  and  allows  them  to 
effectively  share  information  and  cooperate  with  existing  function  modules. 
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Figure  2.  An  example  of  function  modules  and  their  Figure  3.  Basic  structure  of  a  function  module 

states  registered  in  the  situation  awareness  layer 


2.2  The  function  layer 
Function  modules 

In  the  proposed  UCCA,  function  modules  are  specialized  in  various  types  of  tasks  in  UAV  missions.  The  capabilities  of 
a  function  model  are  two-fold.  First  it  acts  as  an  information  processor  that  uses  incoming  data  to  update  its  states. 
Second  it  acts  as  a  controller  guiding  the  UAV,  utilizing  its  onboard  resources  and  cooperating  with  other  function 
modules  to  accomplish  its  objectives. 

Fig.  3  shows  the  basic  structure  of  a  function  module.  Inside  a  function  module,  a  set  of  states  and  parameters,  denoted 
as  Sj> . . .  9  Sf1 ,  summarizes  the  current  information  and  the  state  of  the  specific  problem  that  the  function  module  is  to 

handle.  Table  1  shows  some  examples  of  function  modules  and  their  states.  In  general,  states  and  parameters  of  a 
function  module  are  dynamic  and  stochastic  in  nature,  whose  levels  of  uncertainties  reflect  the  level  of  situation 
awareness  for  the  objective  of  interest. 

To  facilitate  cooperation,  function  modules  have  well  defined  inputs  and  outputs.  Inputs  of  a  function  module  include  i) 
data  for  processing  and  state  updates,  ii)  states  of  other  function  modules  from  the  SAL  that  are  relevant  to  its  task,  iii)  a 


targeting  objective  specified  as  a  requirement  on  a  utility  function  of  the  states,  e.g.,  J(S ?,  Sf  . . .)  >  r. ,  and  iv)  control 
sets  available  for  the  function  module  to  control  the  UAV  or  other  resources. 


Table  1.  Exemplar  states  in  different  function  modules 


Function  module 

Possible  states  and  parameters 

Target  identification 

target  ID,  target  type,  other  target  properties  of  interest,  etc. 

Search 

spatio-temporal  density  of  the  target’s  occurrence,  probability  of  target 
occurrence  in  different  surveillance  regions,  target  detection  results,  etc. 

Tracking 

number  of  targets,  target  spacing,  tracks,  road  map  information,  etc. 

UAV  safe  navigation 

threat  modeling  parameters,  UAV  survival  probabilities,  etc. 

Note  that  in  the  proposed  UAV  control  architecture,  instead  of  maximizing  utility  functions,  objectives  for  function 
modules  are  set  as  requiring  utility  functions  to  reach  a  desired  performance  threshold.  For  example,  it  is  more 
reasonable  to  require  the  tracking  module  to  maintain  the  tracking  accuracy  of  a  target  above  desired  threshold,  rather 
than  to  track  the  target  as  accurate  as  possible  with  all  the  UAV’s  resources.  This  approach  is  a  more  appropriate  for 
UAV  missions  that  have  multiple  objectives  competing  for  UAV  resources.  It  is  adopted  in  the  hierarchical  layered 
decision  framework  (HLDF,  to  be  introduced  later  for  the  TCM)  for  making  Pareto  optimal  control  decisions  with 
desirable  performance  tradeoffs  among  multiple  UAV  tasks. 

Outputs  of  a  function  module  consist  of  i)  the  qualified  sets  of  control  decisions,  and  ii)  sub-objectives  (requirements) 
for  other  function  modules  whose  states  are  relevant  (as  the  function  module’s  inputs).  The  outputs  allow  a  function 
module  to  directly  control  the  UAV  and  its  onboard  resources,  or  indirectly  through  other  function  modules  for  the 
achievement  its  objective.  Note  that,  because  of  the  way  in  which  objectives  for  function  modules  are  specified,  instead 
of  producing  a  single  unique  control  decision  for  the  UAV  or  other  resources,  function  modules  yield  the  sets  of 
qualified  control  decisions,  which  are  subsets  of  the  available  control  sets  from  the  inputs. 

The  task  coordination  module  (TCM) 

A  special  component  of  the  function  layer  is  the  task  coordination  module  (TCM)  which  handles  the  coordination  and 
cooperation  of  other  function  modules.  Function  modules  need  to  share  the  control  of  the  UAV  and  the  onboard 
resources  for  their  assigned  objectives.  For  example,  tracking  and  search  function  modules  use  the  same  set  of  sensors, 
and  both  need  to  control  the  flight  path  of  the  UAV  for  their  respective  objectives.  Thus  control  decisions  for  the  UAV 
and  its  onboard  resources  need  to  balance  among  multiple  objectives  and  cannot  be  directly  made  by  any  single  function 
module.  As  illustrated  in  Fig.  1,  the  physical  layer  of  the  UAV  is  directly  controlled  by  the  TCM  rather  than  other 
function  modules.  One  major  function  of  the  TCM  is  to  cooperate  with  other  function  modules  to  make  control  decisions 
that  yield  desired  overall  system  performance.  For  this  purpose  the  TCM  adopts  a  Hierarchical  Layered  Decision 
Framework  (HLDF)  proposed  in  [1]. 


Q  :  the  full  control  set 
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</2(c,...)  >  t2} 


Figure  4.  Decision  making  process  in  the  HLDF 

Fig.  4  illustrates  the  HLDF,  where  the  control  decisions  are  made  based  on  a  prioritized  list  of  current  objectives 
(including  sub-objectives).  It  starts  from  the  objective  with  the  highest  priority.  The  TCM  first  sends  the  objective 
(denoted  as  and  the  full  control  set  Q,  e.g.,  for  the  UAV,  to  the  corresponding  function  module.  The 

outputs  from  the  function  module  include  a  reduced  control  set  Qi  that  qualified  for  the  requirement  on  Then  the 
TCM  proceeds  with  the  objective  of  highest  priority  in  the  remaining  objective  list,  by  sending  the  objective  with  the 
reduced  control  set,  in  this  case  to  the  corresponding  function  module.  The  process  repeats  and  the  control  set  will 
converge  to  the  final  control  decision  with  the  desired  overall  performance.  Note  that  the  control  set  can  be  discrete  or 
continuous;  the  uniqueness  of  the  final  control  decision  can  be  guaranteed  with  minor  modification  to  the  objective  with 
the  lowest  priority. 

Major  benefits  of  this  approach  include:  i)  it  allows  the  specification  of  desired  performance  requirement  to  each 
individual  function  module,  and  yields  Pareto  optimal  control  decisions  with  desired  overall  system  performance  on 
multiple  objectives;  ii)  an  objective  with  a  higher  priority  will  be  protected  from  possible  compromises  from  other  less 
important  ones;  iii)  fully  compatible  with  modular  structure  in  the  function  layer;  and  iv)  depending  on  the  nature  of  the 
objective,  suitable  control  strategies,  e.g.,  one-step  or  multi-step  look-ahead  control  policies,  etc.,  can  be  used  for 
different  function  modules. 

In  [1]  the  problem  of  the  cooperative  control  of  a  group  of  UAVs  for  a  surveillance  mission  in  a  hostile  environment. 
The  HLDF  was  proposed  for  the  handling  multiple  objectives  in  UAV  missions.  Objectives  in  the  surveillance  mission 
include  i)  navigating  the  UAVs  safely  in  the  hostile  environment  with  stationary  and  moving  threats,  ii)  search  for 
targets,  iii)  target  identification,  and  iv)  maintain  adequate  tracking  accuracy  on  detected  targets.  Fig.  5(a)  shows  the 
scenario  with  a  group  of  4  UAVs,  5  moving  targets,  and  three  restricted  regions  (denoted  by  the  red  circles)  that  the 
UAVs  should  avoid.  In  this  work  to  avoid  making  myopic  control  decisions,  a  multiple  objective  and  a  multi-step  look¬ 
ahead  control  strategy  was  also  proposed  based  on  the  Roll-out  policy  [3]. 
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Figure  5  (a).  UAV  surveillance  scenario 


(b).  Minimum  survival  probability  of  UAV  1 


To  demonstrate  the  advantage  of  the  layered  decision  framework  over  the  conventional  combined  objective  approach, 
Fig.5  (b)  shows  the  minimum  survival  probabilities  of  UAV  1  over  100  Monte  Carlo  simulations  with  each  simulation 
lasting  500  time  intervals.  It  was  shown  that,  for  the  objective  of  safe  navigation,  which  is  to  keep  the  survival 
probability  of  the  UAV  close  to  the  required  threshold  rPS  =0.9 ,  the  layered  decision  framework  (HLDF)  with  a  9-step 

look-ahead  path  decision  strategy  yield  significantly  better  performance  than  two  conventional  combined  objective 
approaches.  With  HLDF,  the  rare  drop  of  the  UAV  survival  probability  to  0.8  occurred  only  once  in  the  100  runs;  while 
the  combined  objective  approaches  failed  to  meet  the  objective  most  of  the  time.  See  [1]  for  the  details  on  the  problem 
formulation,  algorithms  and  simulation  results. 


Another  function  of  the  TCM  is  to  allow  the  cooperation  among  different  function  modules.  As  shown  in  Fig.  2,  function 
modules  are  interconnected  with  states  of  one  (registered  with  the  SAL)  being  the  inputs  of  another.  Fig.  3  shows  that 
outputs  of  a  function  model  may  include  new  objectives  for  other  function  modules  whose  states  are  the  function 
module’s  input.  In  the  proposed  UAV  control  architecture,  a  newly  generated  sub-objective  will  be  added  to  the 
objective  list  of  the  TCM  with  a  priority  level  inherited  or  assigned  form  its  parent  objective,  and  be  appropriately 
dispatched  by  TCM  to  the  corresponding  function  module.  This  mechanism  allows  multiple  function  modules  in  the 
function  layer  work  as  an  integrated  system  for  UAV  missions/tasks. 


3.  SUPPORT  FOR  THE  COOPERATION  OF  A  TEAM  OF  UAVS 

The  proposed  UCCA  provides  efficient  and  effective  support  for  the  cooperative  control  of  a  group  or  groups  of  UAVs 
for  complex  missions.  For  UAV  cooperation,  a  dedicated  networking  module  (NM)  will  be  implemented  in  the  function 
layer  to  allow  UAVs  to  maintain  cooperatively  a  communication  network,  and  provide  communication  services  for  other 
function  modules. 


Figure  6.  Cooperation  of  UAVs  on  various  tasks 


Fig.  6  illustrates  how  function  modules  on  multiple  UAVs  are  able  to  cooperate  within  the  proposed  UAV  UCCA.  For 
cooperation,  communication  requests  are  sent  to  the  networking  module  through  the  TCM.  The  NM  will  take  care  of  the 
transmission/reception  of  data  to/from  other  UAVs.  Received  data  will  be  forward  by  the  NM  to  the  data  reception 
buffer  of  the  corresponding  function  module  in  situation  awareness  layer,  or  be  forwarded  to  the  next  UAV  along  the 
data  package’s  routing  path  in  the  communication  network. 

Networking  Modules  on  UAVs  also  have  the  function  of  maintaining  cooperatively  the  connectivity  and  capacity  of  the 
communication  network  to  meet  dynamic  communication  needs  of  UAV  mission.  Like  other  function  modules,  the  NM 
is  able  to  send  control  requests  to  the  TCM  to  control  the  UAV  according  to  its  networking  objectives.  Through  the 
networking  module,  function  modules  such  as  coordinate  mission  planning,  cooperative  path  planning,  combat,  tracking, 
search,  of  a  UAV  can  communicate  with  their  counterparts  of  other  UAVs,  which  allows  cooperation  of  UAVs  for 
various  tasks.  In  this  proposed  work,  cooperative  control  algorithms  that  are  able  to  adapt  to  network  conditions  will  be 
developed  for  these  function  modules.  In  a  game  theoretical  framework,  cooperative  control  strategies  will  be  developed 
to  allow  tradeoffs  between  cooperation  performance  and  communication  requirements  [6—13]. 


Figure  7.  Coordinated  mission  planning  under  the  proposed  UAV  control  architecture 


For  UAV  cooperation,  the  proposed  UAV  UCCA  also  facilitates  the  integration  of  high  level  function  module  for 
coordinate  mission  planning,  and  allows  it  to  work  effectively  with  existing  lower  level  function  modules.  As  illustrated 
in  Fig.  7,  the  function  module  for  coordinate  mission  planning  allows  a  group  of  UAVs  to  cooperate  over  a  complicated 
mission  by  assigning  the  mission’s  tasks  to  subsets  of  UAVs.  For  a  UAV,  tasks  assigned  from  the  coordinate  mission 
planning  module  (CMPM)  generate  a  dynamic  list  of  objectives.  The  objectives  are  then  dispatched  through  the  TCM  to 
the  corresponding  function  modules  for  execution.  By  monitoring  relevant  states  in  the  situation  awareness  layer,  the 
CMPM  gets  feedback  on  the  degrees  of  achievement  of  the  tasks,  based  on  which  new  objectives  will  be  generated 
towards  the  achievement  of  the  assigned  tasks.  In  addition,  the  CMPM  will  support  dynamic  mission  planning  which 
allows  the  reassignment  tasks  to  UAVs  according  to  information  feedback  from  field  level  operations. 


4.  LEARNING  UNDER  THE  PROPOSED  UAV  UCCA 


The  capability  of  learning  is  crucial  for  UAVs  to  operate  and  accomplish  their  missions  in  dynamic  uncertain 
environments  and  in  the  presence  of  intelligent  adversaries.  In  this  proposed  work,  learning  functions  for  UAV  missions 
are  categorized  into  two  types.  The  proposed  UAV  control  architecture  supports  an  objective  driven  learning  (ODL) 
framework  to  effectively  guide  the  UAVs  to  increase  the  level  of  situation  awareness.  It  involves  not  only  learning 
algorithms  for  processing  data  and  reducing  mission/environmental  uncertainties  but  also  algorithms  that  are  able  to 
evaluate  impact  of  uncertainties  on  various  tasks  and  actively  guide  the  learning  process. 

In  the  proposed  UCCA,  uncertainties  of  the  environment  and  adversaries  are  reflected  by  the  uncertainties  of  states  and 
parameters  in  various  function  modules.  In  the  proposed  ODL  framework,  in  addition  to  learning  algorithms  that  reduce 
uncertainties  in  UAV  missions,  algorithms  will  be  developed  for  function  modules  to  evaluate  the  impact  of 
uncertainties  to  the  achievement  of  their  objectives  and  set  appropriate  objectives  for  learning  and  information  gathering 
processes.  Fig.  8  shows  an  illustrative  example  of  the  ODL  process,  where  a  high-level  function  module  cooperates  with 
several  low-level  function  modules  with  learning  capabilities  and  guides  the  learning  process  in  service  of  its  objective. 


Figure  8.  The  objective  driven  learning  process  in  the  proposed  UCCA 


In  general,  algorithms  for  search,  tracking,  and  target  identification  are  learning  algorithms,  which  process  information 
from  onboard  sensors  to  reduce  the  uncertainty  in  targets’  presence,  kinematic  states,  and  identities  for  the  needs  of  high- 
level  functions  modules  for  combat  identification,  mission  assignment,  etc.  Other  learning  algorithms  involve  the 
learning  of  environmental  parameters  which  allow  algorithms  to  adapt  to  environmental  uncertainties  and  changes,  such 
as  the  learning  of  spatio-temporal  density  parameters  for  event  occurrence  in  a  search  mission  [4,14].  Another  example 
is  the  learning  of  state  transition  costs  in  the  Markov  decision  process  for  UAV’s  mission  plan  [5].  These  algorithms  can 
be  accommodated  in  the  UCCA  under  the  ODL  framework. 


5.  CONCLUSIONS 

In  this  paper,  a  unified  cooperative  control  architecture  (UCCA)  is  presented,  which  supports  effective  cooperation  of  a 
group  of  UAVs  and  learning  capabilities  in  uncertain  dynamic  mission  environments.  It  has  a  concise  three  layered 
structure,  which  consists  of  i)  a  physical  layer  that  defines  the  basic  control  space  of  the  UAV  and  provides  interfaces  for 


the  control  of  the  UAV  and  onboard  resources,  ii)  a  function  layer  that  has  various  function  modules  specialized  in 
different  tasks  in  UAV  missions  and  a  special  task  coordination  module  (TCM)  for  the  coordination  and  cooperation  of 
the  function  modules,  and  iii)  a  situation  awareness  layer  that  contains  states  registered  by  the  function  modules  to 
facilitate  the  sharing  of  information  among  the  function  modules.  To  allow  function  modules  to  efficiently  share  the 
controls  of  the  UAV  and  the  onboard  resources,  the  TCM  adopts  a  novel  hierarchical  layered  decision  framework 
(HLDF),  which  is  able  to  yields  Pareto  optimal  control  decisions  with  desired  overall  system  performance.  To  facilitate 
the  effective  cooperation  among  different  function  modules  a  networking  module  will  be  implemented  in  the  function 
layer,  which  is  dedicated  to  maintaining  a  communication  network  among  the  group  of  UAVs  and  provide 
communication  services  to  other  function  modules.  The  proposed  UCCA  also  facilitates  the  integration  of  higher-level 
function  modules,  such  as  coordinate  mission  planning,  and  allows  them  to  cooperate  effectively  with  existing  lower- 
level  function  modules.  Finally,  an  objective  driven  learning  (ODL)  approach  is  proposed  under  the  UCCA.  It  allows 
UAVs  to  systematically  explore  uncertain  mission  environments  to  increase  the  level  of  situation  awareness  for  the 
achievement  of  their  mission/task  objectives. 
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